Embodiments described herein relate to object storage technology, and more specifically, to methods and an apparatuses for object storage.
Object storage technology is a storage technology for both private clouds and public cloud service providers, and provides redundant, scalable data storage capable of storing petabytes of data using clusters of standardized servers.
In general, an object storage system takes an object as a basic unit of storage, and each object is an integration of data and data property set. When an object is created, metadata is used to identify data properties like name, creation time, owner, creator, size or other user-defined properties. Further, objects are stored in the object storage device. The object storage device can manage the objects stored thereon and metadata associated with these objects.
In the object storage device, a file system can be used as an underlying persistency layer. In such an object storage device, a large number of objects with small size can be usually processed in two ways, to optimize performance of the object storage.
One way is to use a log structured store data file. Such a file itself has a relatively large size, in which a plurality of objects with small size can be stored. Whenever a new object is created, the new object will be appended to the last object of the file, to implement sequential I/O operation for object access, so as to improve I/O efficiency.
The other way is to use an optimized in-memory index, which has a very small memory footprint (approximately 10 bytes per object), and can reduce a relatively large memory footprint (about several hundreds of bytes to 1K bytes per object) generated from metadata storage using inode.
In the log structured store data file, in general, the data (which may also be referred to as content) of the object together with the metadata of the object are stored as a whole. The new object can only be appended to the log structured store data file, so when the object in the file is updated, an update operation usually includes appending the new object to the last object in the log structured store data and deleting the original object. Such update operation does not destroy the sequential I/O operation. However, such update operation does not distinguish metadata update from content update. Therefore, even in the case where only the metadata of the object is updated and the content is not updated, it is still necessary to copy the original content of the object and set new metadata. Typically, the metadata has a size much smaller than that of the content, and therefore, when such update operation is used, copy of the content of the unchanged object cause a wasting of I/O operation and disk space resources.